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ABSTRACT 



An experimental study compared the effects of controlled and 
free writing exercises and of evaluative vs. communicative feedback on the 
writing quality and apprehension of Korean college students of English as a 
foreign language (EFL) . Students in two sections of a freshman reading course 
wrote dialogue journals as practice in free writing, and received 
communicative feedback focusing on meaning and using no grading or error 
correction. Students in two other sections answered adapted comprehension 
questions as practice in controlled writing, and received evaluative feedback 
focusing on formal linguistic features, with grading and error correction. 

Two teachers taught the sections, each teaching one free- writing and one 
controlled-writing section. Changes in writing quality, as measured by 
holistic essay evaluation, and writing apprehension, assessed using an 
objective measure, were investigated. Results indicate that students writing 
dialogue journals and getting communicative feedback improved in writing 
quality and possibly experienced reduced writing apprehension, and that this 
change was greater than that for students practicing controlled writing and 
receiving evaluative feedback. Contains 28 references. (MSE) 
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Abstract 



This experimental study investigated the effect of controlled vs. free 
writing and evaluative vs. communicative feedback on writing quality and 
writing apprehension of Korean EFL college students. Of freshman English 
reading courses, students in two experimental sections wrote dialogue journals 
as a practice of free writing and received communicative feedback, and their 
progress was compared with the progress of students in two control sections 
who answered adapted comprehension questions as a practice of controlled 
writing and received evaluative feedback. 

Two teachers taught the one-semester course, each teaching one section 
each of experimental and control sections. To students' dialogue journal 
entries, the experimenter responded, focusing on meaning, with no grade and 
no error correction; to students' answers to adapted comprehension questions, 
the experimenter responded, focusing on formal linguistic features, with grade 
and error correction. 

Pre- and posttest data were analyzed, using ANOVA, on variables, (1) 
change in writing quality, measured by holistic rating of essay writing tasks, 
and (2) change in writing apprehension, measured by objective rating of the 
Writing Apprehension Scale (Daly & Miller, 1975). 

Results showed that writing dialogue journals and getting communicative 
feedback significantly improved students' writing quality and possibly reduced 
their writing apprehension more than answering adapted comprehension 
questions and getting evaluative feedback. Those results were interpreted as 
supporting the use of free writing and communicative feedback in EFL 
courses. 
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This article reports my experimental study of how controlled vs. free 
writing and different types of feedback affect writing quality and writing 
apprehension of Korean college freshmen. Adapted comprehension 
questionnaires were used for controlled writing, and dialogue journal entries, 
free one. And, to those two kinds of writing, evaluative and communicative 
feedback were given, respectively. In this article, I'll introduce the study by 
discussing its theoretical background and the experiment itself. 



Theoretical Background 

In Korea, English has been taught as a foreign language (EFL) at all 
levels of educational settings, and writing instruction in English begins at the 
college level without any nationwide curriculum for it. However, at most 
colleges, the viewpoint of writing has been that writing is producing, 
involving transmission of the writer's message and making it clear for others. 
Accordingly, the teaching of writing there has put emphasis on the written 
product, and writing itself has almost always been served as an evaluative 
purpose, causing high writing apprehension of most student writers. These 
current viewpoint and teaching practice of EFL writing at most Korean 
colleges correspond to those of ESL writing in American language educational 
contexts especially from 1945 till 1970. 

Then, let me (1) distinguish between EFL and ESL, (2) examine the 
history of ESL writing instruction in the USA since 1945, locating in the 
history each of basic writing instruction and advanced counterpart at a 
Korean college where I conducted my experiment in 1996, and (3) explain the 
background of the experiment. 
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English as a Foreign Language vs. English as a Second Language 
According to Snow (1986), English can be a foreign language when it is 
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taught as one subject in school and used typically only at instructional 
settings (e.g., English in South Korea); English can be a second language 
when it is taught to the speakers of other languages in settings either where 
it is the language of the majority and the media of T.V., radio, and 
newspapers (e.g., English in the U.S.A.), or where it is designated as an 
official language of government and education (e.g., English in the Philippines, 
English in Hong Kong, and, English in South Africa). 

History of ESL Writing Instruction in the U.S.A. since 1945 - 
Controlled Composition Model 

1945 was the beginning of the modem era in second language teaching in 
the USA. From that time to 1985, the principal motto in language education 
was that language was speech and learning was habit formation. Therefore, 
during that period, writing was regarded as a secondary concern, the servant 
of the other language skills. In that context, Silva (1990) discusses that 
controlled composition model dominated ESL writing instruction. According to 
him, it is a formal accuracy-centered approach, in a word, emphasizing the 
correct use of formal linguistic features such as vocabulary, sentence 
structure, and mechanics. Accordingly, its teaching practice focuses on 
imitation and manipulation of model passages, in a text, carefully constructed 
and graded for vocabulary and sentence patterns. On manipulation, a teacher 
instructs substitutions, transformations, expansions, and completions. Students 
write their composition on teacher-given topics adopted from the passages. 
They write, without any sense of audience and purpose, only by imitating and 
manipulating the passage contents. The teacher responds evaluatively to their 
composed product and corrects it, centering on the formal linguistic mistakes. 
Therefore, it is guessed that the students who leam to write English 
O 
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composition by following this model may not improve their writing quality 
sufficiently, for, in measuring writing quality, the other features such as ideas, 
organization, and style are also included. Also, it is assumed that those 
students can become high apprehensive writers due to correction and 
evaluation on their written product According to Faigley, Daly, and Witte 
(1981), high apprehensive writers find writing unrewarding and experience 
more than normal amounts of anxiety when they must write, consequently 
avoiding the situations that require writing. At the Korean college where my 
experiment was conducted, basic writing has been taught in freshman English 
courses for one year, where writing has been considered as subsidiary to the 
other language skills (speaking, listening, and reading). In teaching basic 
writing, most of the teaching aspects of controlled composition model have 
been prevalent. 

ESL Version of the Current- traditional Rhetoric 

Then, in the mid-sixties, there was an increasing awareness that ESL 
students needed to produce extended written discourse and controlled 
composition was not enough for that need. Therefore, a bridge between 
controlled and free writing was necessary. In that context, Silva (1990) again 
contends that ESL version of the current-traditional paradigm directed ESL 
writing instruction. That version of the paradigm adds Kaplan's (1966) theory 
of contrastive rhetoric, which assumes that second language writers would 
adopt a rhetoric and a series of thought which might violate the expectations 
of the native reader, to the current- traditional paradigm in first language 
composition. According to Silva in the same source, it is an organization- or 
format-centered approach, in a word, trying to provide ESL students with 
logical construction and arrangement of discourse forms. Accordingly, its 
teaching practice focuses on paragraph development and essay development of 
model compositions in a text. On paragraph development, a teacher instructs 
O 
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elements of a paragraph (topic sentence, support sentence, concluding 
sentence, and transitions) and its developmental patterns (compare and 
contrast, classification, illustration, exemplification, definition, and cause and 
effect). On essay development, an expansion of paragraph principles to larger 
stretches of discourse, the teacher gives lessons in larger structural entities 
(introduction, body, and conclusion) and organizational patterns (narration, 
description, exposition, and argumentation). Students write their composition 
by internalizing the structure of a given model writing and then applying that 
structural knowledge gained from it to their own, a parallel piece with it. In 
this paradigm also, the teacher responds evaluatively to their composed 
product and corrects it, centering on usage such as syntax, spelling, and 
punctuation, which may cause those students to become high apprehensive 
writers. At the Korean college where my experiment was conducted, advanced 
writing has been taught for the students who already took basic writing in 
freshman English courses. In teaching advanced writing, most of the teaching 
aspects of ESL version of the current-traditional rhetoric have been dominant 
Also, at some other colleges, depending on their own curriculum, English 
writing instruction for their freshmen has been given by following this model. 
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Process-centered Approach to Writing 

However, during the 1970s and the 1980s, controlled composition model 
and the traditional, product-centered paradigm were beginning to crumble, and 
instead, a new, process-centered paradigm was emerging. Scholars (Hairston, 
1982; Marshall, 1987; Raimes, 1985, 1991; Stotsky, 1983; Tierney, 1992; Tierney 
& Shanahan, 1991; Zamel, 1987) assert that writing is a generative and 
cognitive process as a means of thinking, problem-solving, and constructing 
activities, and also, a recursive one in which pre-writing, writing, and 
re-writing overlap and intervene. They continuously argue that therefore, 
writing is not only a communication skill but a way of learning and 
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developing, for, most writers, both skilled and unskilled ones, have only a 
partial notion of what they want to say when they begin to write, and their 
ideas develop in the process of writing. In that context, Silva (1990) maintains 
that the new paradigm, process-centered approach to writing governed ESL 
writing instruction. According to him, it is a meaning -centered approach, in a 
word, declaring that content, ideas, and the need to communicate determine 
form. Accordingly, its teaching practice focuses on brainstorming, free writing, 
self-generated topics, drafting and redrafting, attending to errors only at the 
final stage, small-group activities, and teacher-student one-to-one conference, 
all of which can promote discovery of meaning and ideas. Students work 
through composing processes with their teacher's help - help for strategies 
for getting started (finding topics, establishing audience and purpose, 
generating ideas and information, and planning structure and procedure), for 
drafting (encouraging multiple drafts), and for revising (adding, deleting, 
modifying ideas; rearranging; and editing with attention to vocabulary, 
sentence structure, grammar, and mechanics). The teacher intervenes in their 
composing process, emphasizing it rather than their composed product, and 
responds communicatively, focusing on meaning, to each draft, with no grade. 
For those reasons, it is supposed that the students who learn to write English 
composition by following this approach can become low apprehensive writers. 
According to Faigley, Daly, and Witte (1981), those writers are confident in 
their writing ability, not avoiding the situations that demand writing but 
frequently enjoying it. Also, Faigley et al. in the same source claim that, 
compared with their high apprehensive counterparts, low apprehensive writers 
produce essays significantly longer and syntactically more mature by 
developing their ideas better and putting more information into each 
communicative unit. In a word, it may be said that their writing quality can 
be better than high apprehensive writers'. At most Korean colleges, this 
process -centered approach to writing has barely been tried, and especially, at 
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the college where my experiment was conducted, never, before the 
experimental period. 



Background of the Experiment 

As mentioned above, at most Korean colleges, EFL writing instruction 
has been given by following controlled composition model or ESL version of 
the current-traditional rhetoric in the USA, with evaluative feedback. As a 
result of that, generally, there have been some problems in writing quality 
and writing attitude of most Korean college students: They lack especially 
their own ideas in writing and apprehend a lot for writing act. So, to 
contribute to solve those problems, I thought of introducing, at the Korean 
TESOL conference, free writing and communicative feedback as a modified 
practice of process-centered approach to writing, expecting that the new 
writing mode and feedback type might improve those students' writing quality 
and decrease their writing apprehension. And, before introduction, to see 
whether the expectation would come true or not, I decided to conduct an 
experiment, at a college, with two different writing modes (controlled vs. free 
writing) and feedback types (evaluative vs. communicative feedback). 



Experiment 

Design and Hypotheses 

I conducted the experiment at a Korean college, throughout the first 
semester of 1998-1997 academic year, by using four intact reading sections of 
freshman English. In the experiment, I managed to meet all of Hillocks' (1984; 
1986) criteria for acceptable research conditions, to make it carefully designed 
and controlled, as follows: 

- involving a treatment over some period of time leading to a posttest; 

- making use of a scale of writing quality applied to samples of writing; 
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- exercising minimal control for teacher biases by using two teachers, 
each teaching one section for each treatment; 

- controlling differences between groups of students by statistical analysis! 

- scoring pre- and posttest under conditions which help to assure validity 
and reliability; 

- coding essays to ensure that raters cannot detect treatment and time 
[pre- or posttest]; 

- having two or more raters, their scores summed and averaged; 

- achieving .70 or higher inter-rater reliabilities on holistic scoring. 

By setting up two experimental and two control sections, the experiment 
compared free writing and communicative feedback with controlled writing 
and evaluative feedback in their effect on students' writing quality and 
writing apprehension: For a duration of 14 week meeting period, students in 
experimental sections wrote dialogue journals about their reading contents as 
a practice of free writing and received communicative feedback from the 
respondent, the experimenter, over a ten week period, and their progress was 
compared with the progress of students in control sections who answered 
adapted comprehension questions about the same contents as a practice of 
controlled writing and received evaluative feedback from the same respondent 
over Hie same period. 

To know the effect of each treatment on dependent variables, this 
experiment used a Pretest-Posttest Control Group Design (Campbell & 

Stanley, 1963). Each treatment here was writing dialogue journals and getting 
communicative feedback or answering adapted comprehension questions and 
getting evaluative feedback, and the dependent variables and measurement 
instruments used were: 

- writing quality on pre- and posttest, as measured by writing tasks! 

- writing apprehension on pre- and posttest, as measured by a writing 
apprehension scale. 
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The two hypotheses in the experiment were: 

Writing Quality 

The experimental group, which was exposed to writing dialogue journals and 
receiving communicative feedback as treatment, would show significantly 
higher gains in writing quality, from pre- to posttest, as measured by writing 
tasks that would be judged by experienced English composition teachers, as 
compared to the gains in writing quality shown by the control group, which 
was exposed to answering adapted comprehension questions and receiving 
evaluative feedback as treatment, with the level of significance set at p < .05; 
Writing Apprehension 

The experimental group, which was exposed to writing dialogue journals and 
receiving communicative feedback as treatment, would show significantly 
greater decrease in writing apprehension, from pre- to posttest, as measured 
by the Daly-Miller Writing Apprehension Scale, as compared to the decrease 
in writing apprehension shown by the control group, which was exposed to 
answering adapted comprehension questions and receiving evaluative feedback 
as treatment, with the level of significance set at p < .05. 

Limitations in the Experiment 

There were some limitations in the experiment due to a few constraints 
in the experimental setting. 

- Instead of truly randomizing subjects, I used intact sections, randomized 
only by normal enrollment procedures, since the institutional procedures 
of the college did not permit it 

- I used reading sections, not writing sections, for, freshman English 
courses at the college consisted of speaking and reading section, and 
basic writing instruction had been customarily given in the reading 
section. 

- The allotted time for reading section (2 hours per week) forbade 
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teaching/leaming writing enough in the class. So, the two participating 
teachers just explained briefly to their students in the experiment, in the 
class, about how to work on each writing practice and each feedback 
type they would get; the students worked on their writing as homework. 
Teaching writing briefly in the class and students' working on controlled 
writing as homework had been customary for basic writing instruction at 
the college. 

- To the students' writing homework, not the participating teachers but I, 
the experimenter, gave written response, for, even the teachers' working 
load concerning teaching reading was much for them. Before the 
experiment, usually, teaching assistants in English department at the 
college had responded evaluatively to freshmen's writing homework. 

Subjects and Participating Teachers 

The subjects in the experiment were all female, first-year students at a 
four-year college, aged around 19, who enrolled for the first semester of 
1996-1997 academic year. In Korea, no clear picture of the typical EFL college 
students has been available, since there has been no nationwide standardized 
English proficiency test at the college level. However, the participating 
teachers and I all agreed that the subjects' English proficiency level was 
advanced as compared to the level of students at other colleges in Korea, for, 
the college had had a name-value for its competitiveness at the nationwide 
standardized entrance exam requiring high score. At the college, first-year 
students signed up for freshman English, a required course for them, by their 
choice of major field. Their meeting time and instructor were pre-determined 
by the schedule deputy before the semester began. In the experiment, the 
students who chose to major in pharmacy, law, education, and pre-medicine 
participated by signing up for the four reading sections of freshman English 
involved in it. They signed up during the regular registration time without 
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knowing that they were to participate in the experiment. During the first 
meeting, pretests for writing quality and writing apprehension were given to 
each section. Then, the four intact sections were randomly assigned to either 
experimental or control sections by coin toss. That was possible, since all 
those sections were taught with the same material. Lauer and Asher (1988) 
discuss how that kind of situation can happen. So, pharmacy major and law 
major were assigned to experimental sections, and education major and 
pre-medicine major, to control sections, and then, each treatment began. At 
the outset of the experiment, 207 freshmen made up the initial population and 
each section was around 50 students in size. 

The two female participating teachers had similar academic backgrounds 
(Ph.D. or ABD in English) and teaching experiences in reading (3 years, 
both), and satisfactory past performances considered from their tenure-board. 
For the experiment, both taught the one-semester course, each teaching one 
section each of the experimental and the control sections: Teacher 1 taught 
pharmacy major and education major, and teacher 2, law major and 
pre-medicine major. Therefore, based on those conditions — their prior 
education, experience and ability in teaching reading, and each teacher 
teaching one section of each treatment, it was felt that the teacher effect 
would not endanger the validity of the experiment. 

The responsibilities of the teachers were: 

- to attend a few hours of pre- treatment orientation during which the key 
concepts in this experiment — controlled vs. free writing, answering 
adapted comprehension questions vs. writing dialogue journals, evaluative 
vs. communicative feedback — were explained by myself, the 
experimenter; 

- to teach two reading sections of freshman English, one section as an 
experimental one, explaining about how to work on dialogue journals and 
communicative feedback, and the other section as a control one, 
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explaining about how to work on adapted comprehension questions 
and evaluative feedback; 

- to attend some informal meetings with the experimenter during the 
semester to describe what was going on in their classrooms and their 
students' attitude toward each writing homework; 

- to administer two kinds — writing task and writing apprehension 
scale — of pre- and posttest within allotted time for each. 

The teachers agreed that the purpose of the experiment was to compare 
the effects of two different writing practices and feedback types in reading 
sections, and both treatments deserved to be tried in an identical manner. 
None of them were informed of the expected results of the experiment. 

Procedures 

Classes met for 100 minutes once a week. For both their sections, not to 
endanger the reliability of the experiment, the two teachers taught with the 
same instructional mode, the Grammar-Translation Method (Larsen-Freeman, 
1986), followed the same systematic sequence in the course syllabus for 
reading lessons, and assigned the same course requirements except writing 
homework. Concerning teaching/leaming writing, as mentioned before, the 
teachers explained only briefly about how to work on dialogue journals or 
adapted comprehension questions and each feedback type in the class, and the 
students in this experiment worked on either of them as their homework. In 
this one semester-length experiment, both experimental and control group 
students met for a duration of fourteen weeks and each group students wrote 
six dialogue journal entries (see Appendix 1) or answered six adapted 
comprehension questionnaires (see Appendix 2) about the same six reading 
chapters. Those chapters were covered through eleven weeks, since five of 
them were taught, each for two weeks, and one of them, for one week. 
Experimental group students wrote their entries before they began to learn 
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each of those chapters, or in the middle of their learning process, or after 
they learned it They were told that the journal entries should be at least 
three sentences in length (Staton, Shuy, Peyton, & Reed, 1988). Here, writing 
dialogue journals can be a practice of free writing, for, those students wrote 
their entries about topics chosen by themselves from the reading contents, 
freely with their own ideas. To those entries, I, the experimenter, gave 
communicative feedback by responding solely to meaning, not correcting 
errors and not grading the entries. Meanwhile, control group students wrote 
their answers to adapted comprehension questions after they learned each of 
those chapters. Here, answering adapted comprehension questions can be a 
practice of controlled writing, for, those students made their answers to the 
questions, given by myself, adapted from the reading contents, by imitating 
and manipulating the contents. To those answers, I gave evaluative feedback 
by correcting their formal linguistic mistakes and grading. At the end of each 
class when their students were to submit writing homework, the two teachers 
collected and gave to me those students' dialogue journal entry or adapted 
comprehension questionnaire including answers. Then, after giving feedback, I 
gave back to the teachers before each following class and they returned to 
their students during it the entries or the questionnaires including answers. 
These procedures continued throughout one semester. 

Data Collection 

To know the effect of each treatment on writing quality and writing 
apprehension of the students in both groups, I had the two teachers 
administer to all of them pre- and posttest, before and after treatment, with 
two writing tasks and two writing apprehension tests. 

For the writing tasks (see Appendix 3), there were four topics — my 
dream in the past, friendship in my life, my hope for the future, and filial 
piety in my life, and the expected rhetorical form of the tasks on those topics 



O 




15 



13 



was narrative. Of the topics, two were categorized into form A and the other 
two, form B, by coin toss, for those students to have a topic choice for each 
test. The two forms were also randomly crossed by coin toss for them to get 
one form for the pretest and the other form for the posttest. For each test, 
the students in both groups, got their two writing topics one week before 
they wrote, being permitted to jot down ideas or make outlines about their 
own writing task and to use them during actual writing. This followed some 
scholars' (Emig, 1971; Pianko, 1977; Sanders & Littlefield, 1975) claim that it 
is important to involve students in a writing task and also Bridwell's (1979) 
argument that it seems to increase students' involvement to give them time 
for thinking about their writing task before actually writing it Each test was 
taken during the class time, since most theorists (Cooper, 1977; Diederich, 
1974) believe that a direct sample is the best way to measure writing ability, 
and the two teachers supervised each writing session by giving the same 
instructions and time limit (30 minutes). For each writing session, the 
students were provided with papers on which they put their name, major field 
of study, and date for the test before starting to write. All of their written 
papers were collected as soon as each test was done. 

For the writing apprehension tests, the Writing Apprehension Scale 
developed by Daly and Miller (1975) (see Appendix 4) was used in English 
version for both pre- and posttest. However, before the experiment, the Scale 
had never been used with those students either in English or in translation 
into Korean. For each test also taken during the class time, the two teachers 
again supervised by giving the same instructions and time limit (15 minutes), 
and the students were provided with questionnaires on which they put their 
name, major field of study, and date for the test before starting to work on 
them. All of their questionnaires including answers were collected as soon as 
each test was done. The Writing Apprehension Scale (Daly & Miller, 1975) 
consists of 26 questions, 13 positive statements about writing (e.g., "I like 
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seeing my thoughts on paper") and 13 negative statements about writing (e.g., 
"I don't think I write as well as most other people"), responded to on 
five-point Likkert-like scale ("strongly agree," "mildly agree," "neutral," 

"mildly disagree," "strongly disagree"). The Scale has been scored on a 
five-point scale to give high apprehension a high number and low 
apprehension a low number. Thus, a "strongly agree" response to a negative 
question is scored as 5, and a "strongly agree" response to a positive question 
is scored as 1. Accordingly, the scoring gives a point range from 26 to 130; 
arbitrarily, as Reigstad (1991) states, scores of 78 or below (78 would reflect 
a "neutral" response to all questions) are considered to reflect "low 
apprehension," scores of 79 to 99, "high apprehension," and scores above 100, 
"very high apprehension". Daly (1985) discusses that the Writing Apprehension 
Scale has been widely used in composition research, and writing apprehension 
scores have been shown to correlate with a variety of other measures, from 
promptness in turning in assignments to choice of major based on the writing 
demands of the field. Five factors make the Scale a useful measuring device 
for the experiment undertaken here — its face validity as a measure of writing 
attitude! the internal reliability of the test! its correlation with other measures! 
its broad use in composition research, and its ease of administration. 

Data Analysis 

Scoring of the writing samples and the writing apprehension tests was 
done only for the 166 students who met the following conditions'- 

- had neither learned English outside of Korea nor had foreign-bom 
instructors who might have used free writing and communicative 
feedback to teach writing before the experiment; 

- either wrote dialogue journal entries or answered adapted comprehension 
questionnaires more than five times throughout the experiment. 
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Two writing samples were collected from each of the 166 students, and all 
the samples were typed to avoid handwriting effect in scoring. To measure 
the quality of those samples, holistic scoring was used, which is a single 
qualitative holistic judgment based on the factors of ideas, organization, style, 
vocabulary, and sentence structure! as an assessment measure, a 6-point scale 
(see Appendix 5) was adopted, which has been used in the TWE (Test of 
Written English). Two raters for the samples were carefully chosen by 
matching their academic background, teaching experience in ESL writing, and 
opinion about the nature of good writing, to achieve inter-rater reliability. 

They were not informed of the design or the purpose of the experiment; they 
were just asked to read for an overall impression of each writing sample 
(Cooper, 1977), valuing the communicative aspect of it, and also to "reward a 
student for what the student does well" (adopted from the ETS's Advanced 
Placement Examination - XXX, 19XX). For the scoring sessions, all the 
samples were appropriately coded and randomly mixed, so that none of the 
raters knew whether a sample came from the pretest or the posttest section 
or from a student in an experimental group or a control group. While scoring, 
each of the raters rated 332 writing samples in three sessions to avoid a loss 
of reliability due to fatigue. Also, during each session, they freely availed 
themselves with breaks from their work to avoid exhaustion, frequently 
reviewing previously scored samples of their own to maintain consistency in 
their own judgments over time. Thus, all the samples were read by two 
raters, well trained and experienced, and each sample received two scores. On 
the 28 samples where the two raters disagreed by more than one point, the 
third rater, who had the same conditions as a rater as those of the two raters 
and knew nothing about the nature of the experiment, arbitrated by scoring 
them. In that case, this experiment accepted the closest two scores from those 
three scores. Also, the middle score of the three scores was dropped when the 
scores of the other two disagreeing raters proved to be equidistant from that 
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score, for the intent of this process was to get the greatest divergence 
between a pair of holistic scores for a given writing sample. Accordingly, 
every sample received the two scores to which at least two raters agreed 
within one point difference. After all the samples were scored, inter-rater 
reliability was calculated by using Kendall's revision of the Spearman rank 
order correlation coefficient (Kendall, 1948). This recalculation produced an 
inter-rater reliability of .83, a figure above the generally accepted minimum of 
.70 required to insure that ratings from holistic scoring are reliable (Hillocks, 
1984, 1986). 

Also, two ■writing apprehension questionnaires including answers were 
collected from each of the 166 students, and scoring of them by myself was 
mechanical and objective. 

When the scoring of the two kind data was completed, whole group data 
for each kind were averaged for the whole group gains and the gains were 
subjected to statistical analysis for significance set at p < .05, using ANOVA. 

Results 

This section presents statistical analysis of the followings^ 

- the effect of each treatment on writing quality, as gathered from 
holistically rated writing samples; 

- the effect of each treatment on writing apprehension, as gathered from 
the Writing Apprehension Scale (Daly & Miller, 1975). 

Analysis of Writing Quality bv Treatment Based on Holistic Ratines 

From the scoring process for the writing samples mentioned above, two 
pretest and two posttest half scores came out for each of the 166 students. 

For statistical procedure, both the pretest and the posttest half scores were 
combined, which resulted in a possible scale of 2 to 12, low to high possibility 
for the pretest as well as the posttest rating for each of them. The mean 
O 
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changes from pretest to posttest for experimental and control group were 
+0.559 and +0.333, respectively, an improvement in both cases. Table 1 
indicates the mean changes in writing quality, pretest to posttest, for the two 
groups (Insert Table 1 here). 

Table 2 presents an overview of the F test of significance for writing 
quality by treatment based on the holistic ratings. In this case, the treatment 
effect was statistically significant, favoring the experimental group, F(2, 166) = 
5.0574, p = .0259 (Insert Table 2 here). 

Analysis of Writing Apprehension bv Treatment Based on Objective Ratings 

From the scoring process for the writing apprehension tests mentioned 
above, one pretest and one posttest score came out for each of the 166 
students. The mean changes from pretest to posttest for experimental and 
control group were -4.844 and -1.368, respectively, a reduction in both cases. 
Table 3 indicates the mean changes in writing apprehension, pretest to 
posttest, for the two groups (Insert Table 3 here). 

Table 4 presents an overview of the F test of significance for writing 
apprehension by treatment based on the objective ratings. In this case, while 
the treatment effect favored the experimental group, it was not statistically 
significant, F(2, 166) = 2.6597, p = .1048 (Insert Table 4 here). 

Conclusions and Implications 

This section draws conclusions from the data analysis, offering 
implications for teachers and researchers. 

Conclusions 

The first hypothesis in this experiment concerning writing quality was 
supported by the analyzed data which showed that the treatment effect was 
statistically significant, favoring the experimental group, p = .0259. Therefore, 
it was suggested that writing dialogue journals as a practice of free writing 
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and receiving communicative feedback improved students' writing quality 
significantly more than answering adapted comprehension questions as a 
practice of controlled writing and receiving evaluative feedback. 
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The second hypothesis in this experiment concerning writing apprehension 
was not supported by the analyzed data which showed that while the 
treatment effect favored the experimental group, it was not statistically 
significant, (p = .1048). Therefore, it was suggested that the greater decrease 
achieved by the experimental group might have happened by chance. 

However, it could be noted that, in reducing students' writing apprehension, 
writing dialogue journals as a practice of free writing and receiving 
communicative feedback appeared to be at least as effective as or possibly 
better than answering adapted comprehension questions as a practice of 
controlled writing and receiving evaluative feedback. 

Implications for Teachers 

It was supported that free writing and communicative feedback could be 
used in EFL reading courses to improve students' writing quality and 
decrease their writing apprehension more effectively. 

Implications for Further Research 

First, besides dialogue journals used in this experiment, there are various 
kinds of journals that might be used as a practice of free writing in subject 
courses, for example, learning journals and response journals. The common 
features which these two journals share with dialogue journals are: 

- they are not corrected for errors but responded for meaning; 

- they are generally informal in their content, format, and language; and 

- they are not individually graded, maybe graded as a whole later. 

The specific features of these two journals are: 

- for learning journals, generally peers respond briefly in the margin and a 
teacher may respond later, and topics include many things; and 
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- for response journals, generally peers respond briefly in the margin and 
a teacher may respond later as for learning journals, however, students 
respond only to their reading in this case. 

Therefore, it would be useful to implement the other kinds of journals as a 
classroom procedure and to test their effectiveness. 

Second, there may be various kinds of EFL context where free writing 
and communicative feedback can be used and tested as to their usefulness, 
besides the specific EFL context — freshman English reading course at a 
Korean college — where this experiment was conducted. 

Third, there may be various controls — students' gender, students' choice 
of major, teachers' gender, instructional mode, and schools with different 
academic proficiency — which further studies can consider. 

Finally, concerning the issue of general izability for the results in this 
experiment, more data from larger population having more controls are needed 
for making those results more reliable. 
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Table 1 

Analysis of Writing Quality Mean Change Pretest to Posttest bv Treatment 



Treatment 


Pretest 


Posttest 


Change 


Experimental Group 


1.933 


2.492 


+0.559 


Control Group 


1.914 


2.247 


+0.333 



Table 2 








Test of Sienificance for Holistically Scored Writine bv Treatment 










Source 


df 


F 


Sig of F 


between groups 
within groups 


1 

164 


5.0574 


.0259* 



*D < 05. 
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Table 3 

Analysis of Writing Apprehension Mean Change Pretest to Posttest bv 
Treatment 



Treatment Pretest Posttest Change 



Experimental Group 81.411 76.567 -4.844 

Control Group 77.605 76.237 -1.368 



Table 4 

Test of Significance for Writing Apprehension bv Treatment 



Source 


d£ 


F 


Sig of F 


between groups 


1 


2.6597 


.1048 


within groups 


164 







*B < .05. 
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Appendix 1 



A Sample of Experimental Group Subject's Dialogue Journal Writing 

Note: Student journal entry is printed flush left. Teacher response is indented 
5 spaces. Marginal comments by the teacher are inserted in the text, and 
bracketed. 



After reading "Professions for Women" 



What does mean "the Angel in the House"? and How should live women 
at present? 

The past days, excellent <What do you mean by this? You mean 
traditionally good?> women's features are typical. Women should had been 
silent and neat. She has to had concealed her opinion and had obeyed men's 
thoughts. She has to had sacrificed for men and her family all the time. In 
other words, women didn't have their just rights as the human beings. <No, 
they didn't.> 

But in these days, women's social standings improve and their rights are 
protected. Finally we admit that women have a great personality, <ability?> 
therefore they can become independent for their own lives. As the result, 
many women can advance to the society. In recent years women who follow 
occupations have remarkably increased and they achieve their social status 
and self-fulfillments. 

In this situation, how should live women for their future? Above all, 
women should make efforts for their developments. Modern society tends to 
recognize the capable human, so women has to develop their abilities and give 
full play to them for their social advances. Todays women's just rights don't 
be obtained without doing anything. It is acquired through the struggles and 
pains by the women who lived at past. Therefore women who lived at 
present time need to feel some responsibilities and they should work hard to 
their part for women's bright future. <What is women's bright future, 
specifically and concretely speaking?> 



Euiyoung, you've written very enthusiastically and seriously. I like that. 
Also, your handwriting is very beautiful. However, I want you to write 
more personally, giving some examples taken from your family and your 
own experience. 



Appendix 2 



A Sample of Control Group Subject's Adapted Comprehension 

Question-Answers 

Note: Comprehension questions are printed flush left. Student answers are 
indented 5 spaces. Teacher's grading and error correction are not marked in 
this print. 



Comprehension Questions on "Professions for Women" 
Department: Name: 



Please answer with full sentences in English for each question. 



1. What are the characteristics of "The Angel in the House" and how does 
it have to do with professional women? 

She was intensely sympathetic and immensely charming. She was utterly 
unselfish. She excelled in the difficult arts of family life. She sacrificed 
herself daily. In short she was so constituted that she never had a mind 
or wish of her own, but preferred to sympathize always with the mind or 
wishes of others. Above all she was pure. Her purity was supposed to be 
her chief beauty - her blushes, her great grace. Professional women 
should need to do battle with "the Angel in the House". It bothers them 
and wastes their time and so torments them that at last they kill it. 



2. According to Virginia Woolf, what are the two experiences of her own 
professional life? 

The first - killing the Angel in the House - she thinks she solved. "The 
Angel in the House" died. But the second, telling the truth about her own 
experiences as a body, she doesn't think she solved. 



3. How does the author express those women who practise professions and 
what are the next steps for them? 

They have still many ghosts to fight - many prejudices to overcome. 
There are many phantoms and obstacles, looming in her way. To discuss 
and define them is Woolf think of great value and important. It is 
necessary also to discuss the ends and the aims for which we are 
formidable obstacles. 
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Students' Writing Samples 

Sample of Highly -Rated Essay (4 by the first rater, 3.5 by the second rater) 



My Dream in the Past 

Hello. Let me introduce my dreams in the past to you, please. In fact, It 
is very hard to say "my dreams in the past." What do you suppose the 
reason is? The reason is that when I look back upon the past, it is very 
exciting that I had so many dreams. 

When I was very small, I want to be a famous novelist. It seemed to me 
very interesting and wonderful to write a novel. Though you may not believe, 
I wrote 2 novels by myself at 8. Both were tragedies. My mother praised me 
and said that it was very fresh. Then I believed it and I was so happy. But 
the dream changed and I find out another great job. 

My second dream was to be a journalist At 14, I was interested in 
reading daily newspaper. Journalists looked very smart and chic. An active 
carrer woman as a journalist was my ideal. But it also changed when I 
became the middle school student. 

Everything has changed. I realized that I like mathematics and science 
more than writing. Especially, I loved physics. It appeared great. A wise 
physicist like Einstein held the top place in my dream at 16. But to be a 
physicist was too hard for me. I knowed the fact very well. So the dream did 
not last long, either. 

At 17, I became a high school student. Everything has changed again. 

My interest changed, too. In my chemistry and Biology class, I was so 
excited and I was so happy. Then I decided to be a medician. It was the best 
job for me in my high school days. Nevertheless, I took the entrance 
examination last winter, and my mark was unsatisfactory. I can't enter the 
medical college. So I choose the department of pharmacy. Pharmacy was also 
interesting as medicine. 

Now, I am a student of the department of pharmacy of Ewha University. 

I like my subject very well. I believe that I will be a good pharmacist 
someday. 



Sample of Middle-Rated Essay (3 by the first rater, 2 by the second rater) 



My dream in my past 

Until I was fourth grade in the elementary school, I hoped to be a nurse. 
Because, all my friends hoped to be a nurse themselves. Perhaps, the dream 
to be a nurse are common to very young girls. I began learning to play the 
piano when I was five years old. I was interested in playing the piano very 
much. So, I thought I will become a great pianist. 

But, I gave up my dream. Because I entered the middle school, I was very 
busy studying hard. 

Since then, I couldn't make up my mind about future. 

Except studying, I could not think others. 

By being third grade in the high school, I could decide that I will become a 
doctor. In fact, I never wanted to be a doctor. Because I was afraid of the 
blood. But, I wanted to study about DNA, cancer, psychology and AIDS. 
Finally I had no choice but to choose the medical science college. 

Now, I hope to be a doctor sincerely. Though I am afraid of the blood a little, 
I want study about a human-being. And I wish that my dream won't change 
again. 
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Sample of Low-Rated Essay (1.5 by the first rater, 0.5 by the second rater) 



My dreams in the past 

I had many dreams when I was young but I hadn't special dream that I 
would get accomplished something in my future. Anytime I played piano hard 
for pianist What time I drowed the picture for artist and doctor, scientist, 
musicial and so on. like that, Vaguely I used to forecast my future but the 
important thing, I wouldn't think that I will be a teacher. Other side I had 
dream that I must made happy family like everybody, for example my 
husbant have a special work, he also is sincery, kind and lovely and my 
children are good like that, when I was young Vaguely I thought that I 
would have a good family and I would have a special job, after married I 
wish I don't rely on someone (parents, husband). But now, I studied hard for 
teacher, nothing I did think when I was young and nobod will know my 
married life, like that My dreams in my past is beautiful itselves, when I was 
young That I had so dreams and thought was beautiful to me. 
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Appendix 4 



A Writing Survey 

The survey below consists of several statements about writing. For each 
statement, place an X through the response that best describes your feeling, 
using the following code: 

SA: strongly agree 

MA: mildly agree 
N: neutral, no opinion 
MD: mildly disagree 
SD: strongly disagree 

1. I look forward to writing down my ideas. 

SA MA N MD SD 

2. People seem to enjoy what I write. 

SA MA N MD SD 

3. I'm nervous about writing. 

SA MA N MD SD 

4. It's easy for me to write a good composition. 

SA MA N MD SD 

5. I'm no good at writing. 

SA MA N MD SD 

6. I like to write my ideas down. 

SA MA N MD SD 

7. I feel confident in my ability to clearly express my ideas in writing. 

SA MA N MD SD 

8. I expect to do poorly in composition classes before I even enter them. 

SA MA N MD SD 

9. I avoid writing. 

SA MA N MD SD 

10. I am afraid of writing essays when I know they will be evaluated. 

SA MA N MD SD 
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11. I enjoy writing. 

SA MA N MD SD 

12. I would enjoy submitting my writing to magazines for evaluation and 
publication. 

SA MA N MD SD 

13. Discussing my writing with others is an enjoyable experience. 

SA MA N MD SD 

14. I never seem to be able to clearly write down my ideas. 

SA MA N MD SD 

15. I have no fear of my writing being evaluated. 

SA MA N MD SD 

16. My mind seems to go blank when I work on a composition. 

SA MA N MD SD 

17. When I hand in a composition, I know I'm going to do poorly. 

SA MA N MD SD 

18. Taking a composition course is a very frightening experience. 

SA MA N MD SD 

19. Expressing ideas through writing seems to be a waste of time. 

SA MA N MD SD 

20. I don't like my compositions to be evaluated. 

SA MA N MD SD 

21. Writing is a lot of fun. 

SA MA N MD SD 

22. Handing in a composition makes me feel good. 

SA MA N MD SD 

23. I don't think I write as well as most other people. 

SA MA N MD SD 

24. I like seeing my thoughts on paper. 



25. I like to have my friends read what I've written. 



SA MA N MD 

26. I have a terrible time expressing my thoughts 
SA MA N MD 




SD 

on paper. 
SD 
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Appendix 5 



EDUCATIONAL TESTING SERVICE 



TOEFL WRITING TEST 
SCORING GUIDELINES 



Readers will assign scores based on the following scoring guide. Though 
candidates are asked to write on a specific topic, they may treat parts of the 
topic by implication. 

Students should be rewarded for what they do well. 

The followings are typed in the order of (1) general characteristics, (2) 
organization/development, (3) unity/coherence/progression, (4) syntax and 
usage, and (5) word choice for 6- to 1 -point writing samples, each. 



- a 6-point writing sample 

(1) Clearly demonstrates competence in writing on both rhetorical and 
syntactic levels though it may have occasional errors 

(2) Well organized and well developed; Uses appropriate details to support a 
thesis or illustrate ideas 

(3) Shows unity, coherence, and progression; Effectively addresses the 
writing task 

(4) Syntactic variety; Displays consistent facility in the use of language 

(5) Appropriate word choice 



- a 5-point writing sample 

(1) Demonstrates competence in writing on both rhetorical and syntactic 
levels though it will have occasional errors 

(2) Generally well organized and well developed; May have fewer details 
than 6-point samples 

(3) Shows unity, coherence, and progression; May address some parts of the 
task more effectively than others 

(4) Some syntactic variety 

(5) Some range of vocabulary 



- a 4-point writing sample 

(1) Demonstrates competence on both rhetorical and syntactic levels 

(2) Adequately organized; Uses details to support a thesis or illustrate ideas 

(3) Addresses the writing task adequately but may slight or ignore parts of 
the topic 

(4) Adequate but inconsistent facility; May contain some serious errors 
which occasionally obscure meaning 



- a 3-point writing sample 

(1) Clearly demonstrates some developing competence in writing but it is 
flawed on either rhetorical or syntactic level 

(2) Inadequately organized or developed; Fails to support or illustrate 
generalizations with appropriate or sufficient detail 

(4) Simplistic sentence structure! A pattern or accumulation of errors 

(5) Frequent inappropriate choice of words 
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- a 2-point writing sample 

(1) Suggests incompetence in writing 

(2) Disorganized or very little developed; Little or no detail, or irrelevant 
specifics 

(4) Serious and frequent errors in mechanics, usage, or sentence structure 



- a 1 -point writing sample 

(1) Demonstrates incompetence in writing 

(2) Severely underdeveloped or no response at all 

(3) May be illogical or incoherent 

(4) Serious and persistent writing errors 



Authored by the TOEFL staff at Educational Testing Service 
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